Discriminatory Lossy Source Coding: Side 

Information Privacy 

Ravi Tandon, Member, IEEE, Lalitha Sankar, Member, IEEE, and H. Vincent 

Poor, Fellow, IEEE 



Abstract 

A lossy source coding problem is studied in which a source encoder communicates with two 
decoders, one with and one without correlated side information with an additional constraint on the 
privacy of the side information at the uninformed decoder. Two cases of this problem arise depending on 
the availability of the side information at the encoder. The set of all feasible rate-distortion-equivocation 
tuples are characterized for both cases. The difference between the informed and uninformed cases and 
the advantages of encoder side information for enhancing privacy are highlighted for a binary symmetric 
source with erasure side information and Hamming distortion. 

Index Terms 

lossy source coding, information privacy, side information, equivocation, discriminatory coding, 
informed and uninformed encoders, Heegard-Berger problem, Kaspi problem. 

I. Introduction 

Information sources often need to be made accessible to multiple legitimate users simulta- 
neously, some of whom can have correlated side information obtained from other sources or 
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from prior interactions. A natural question that arises in this context is the following: can the 
source publish (encode) its data in a discriminatory manner such that the uninformed user does 
not infer the side information, i.e., it is kept private, while providing utility (fidelity) to both 
users? Two possible cases arise in this context depending on whether the encoder is informed or 
uninformed, i.e., it has or does not have access to the correlated side information, respectively. 

This question is addressed from strictly a fidelity viewpoint by C. Heegard and T. Berger in [1], 
henceforth referred to as the Heegard-Berger problem, for the uninformed case and by A. Kaspi 
[2], henceforth referred to as the Kaspi problem, for the informed case wherein they determined 
the rate-distortion function for a discrete and memoryless source pair. Using equivocation as the 
privacy metric, we address the question posed above using the source network models in [1] and 
[2] with an additional constraint on the side information privacy at the decoder without access 
to it, i.e., decoder 1 (see Fig. [I]). 

We prove here that the encoding scheme for the Heegard-Berger problem achieves the minimal 
rate while guaranteeing the maximal equivocation for any feasible distortion pair at the two 
decoders when the encoder is uninformed. Informally speaking, the Heegard-Berger coding 
scheme involves a combination of a rate-distortion code and a conditional Wyner-Ziv code 
which is revealed to both decoders. Our proof exploits the fact that conditioned on what is 
decodable by decoder 1, i.e., the rate-distortion code, the additional information intended for 
decoder 2, i.e. the conditional Wyner-Ziv bin index, is asymptotically independent of the side 
information, Y (see Fig.Q]). Observing that the generation of the conditional Wyner-Ziv bin index 
is analogous to the Slepian-Wolf binning scheme, we prove this independence property for both 
the Slepian-Wolf and the Wyner-Ziv encoding. Next, we prove a similar independence property 
for the Heegard-Berger coding scheme, which in turn allows us to demonstrate the optimality 
of this scheme for the problem studied in this paper. 

On the other hand, for the informed encoder case, we present a modified coding scheme (vis-a- 
vis the Kaspi scheme) which achieves the set of all feasible rate-equivocation pairs for the desired 
fidelity requirements at the two decoders. The Kaspi coding scheme exploits the encoder side 
information Y (see Fig. [U via a combination of a rate-distortion code, intended for decoder 1, 
and a conditional rate-distortion code, intended for decoder 2, which is then revealed to both the 
decoders. However, conditioned on what is decodable by decoder 1, i.e., the rate-distortion code, 
the conditional rate-distortion code does not explicitly ensure the asymptotic independence of the 
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resulting index with the side information Y, and therefore, does not simplify the equivocation 
computation at decoder 1. To resolve this difficulty, we present a two-step encoding scheme in 
which the first step is the same as in the Kaspi problem while in the second step we first choose 
the codeword intended for decoder 2 and then bin it. We prove that the resulting conditional bin 
index is asymptotically independent of the side information Y. 

The last part of our paper focuses on a specific source model, a binary equiprobable source 
X with erased side information Y (with erasure probability p) and Hamming distortion con- 
straints. For this source pair, we focus on the rate-distortion-equivocation tradeoffs for both the 
uninformed and informed cases. 

For the uninformed encoder case, we prove that the maximal equivocation is independent 
of the fidelity requirement D 2 at decoder 2, i.e., the only information leaked about the side 
information is a direct consequence of the distortion requirement at decoder I. We also explicitly 
characterize the rate-distortion-equivocation tradeoff for this problem over the space of all 
achievable distortion pairs. Our results clearly demonstrate the optimality of the Heegard-Berger 
encoding scheme from both rate and equivocation standpoints. 

In contrast, for the informed encoder case, we explicitly demonstrate the usefulness of encoder 
side information. We first prove that the set of distortion pairs for which perfect equivocation 
is achievable at decoder 1 is strictly larger than that for the uninformed case. We prove this by 
showing that the informed encoder uses the side information Y via a single description which 
satisfies the distortion constraints at both the decoders while simultaneously achieving perfect 
privacy at decoder 1. Furthermore, we also demonstrate that access to side information leads 
to a tradeoff between rate and equivocation. To guarantee a desired equivocation, we show that 
the minimal rate required can be strictly larger than the rate-distortion function for the original 
Kaspi problem. 

The problem of source coding with equivocation constraints has gained attention recently 
[3]— [13]. In contrast to these papers where the focus is on an external eavesdropper, we address 
the problem of privacy leakage to a legitimate user, i.e., we seek to understand whether the 
encoding at the source can discriminate between legitimate users with and without access to 
correlated side information. Furthermore, our results on the rate-distortion-equivocation tradeoff 
for a binary symmetric source with erased side information for both the informed and uninformed 
encoder cases allow a clear comparison of the results for the same models without an additional 
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Fig. 1. Source network model. 



privacy constraint as studied in [14] and [15]. 

The paper is organized as follows. In Section [III we present the system model. In Section [TTTl 
we first prove the asymptotic independence of the bin index and the decoder side information in 
the Slepian-Wolf and Wyner-Ziv source coding problems. Subsequently, we establish the rate- 
equivocation tradeoff regions for both the uninformed and informed cases. In Section |IVj we 
characterize the achievable rate-distortion-equivocation tradeoff for a specific source pair (X, Y) 
where X is binary and Y results from passing X through an erasure channel. We conclude in 
Section [V] 

II. System Model 

We consider a source network with a single encoder which observes and communicates all 
or a part (X n ) of a discrete, memoryless bivariate source (X n , Y n ) over a finite rate link to 
decoders 1 and 2 at distortions D x and D 2 , respectively, in which decoder 2 has access to Y n 
and an equivocation E about Y n is required at decoder 1. The network is shown in Fig. Q] 
where the two cases with and without side information at the encoder correspond to the switch 
S being in the closed and open positions, respectively. Without the equivocation constraint at 
decoder 1, the problems with the switch in open and closed positions, are the Heegard-Berger and 
Kaspi problems for which the set of feasible (R, D 1 , D 2 ) tuples are characterized by Heegard 
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and Berger [1] and Kaspi [2], respectively. We seek to characterize the set of all achievable 
(R, Di, D 2 , E) tuples for both problems. 

Formally, let (X,y,p(x,y)) denote the bivariate source with random variables X e X and 
Y E y. Furthermore, let X 1 and X 2 denote the reconstruction alphabets at decoders 1 and 2, 
respectively, and let d\ and d 2 such that 

d k : X x X ->■ [0,oo), k — 1,2, (1) 

be distortion measures associated with reconstruction of X at decoders 1 and 2, respectively. 
Let S take the values and 1 to denote the open and closed switch positions, respectively. An 
(n, M, D 2 , E) code for this network consists of an encoder 

/: X n x S-y n ^J = {1,...,M} (2) 



and two decoders, 



9l :{l,...,M}^X?, and 



n 
2 • 



32 :{l 1 ...,M}xf4l 

The expected distortion D k at decoder k is given by 

1 n / - \ 

Djk = E-Y^d k (Xi,Xi) , k = 1,2, (3) 
n i=1 \ J 

where ^ = g 1 (f (X n )), X 2 = g 2 (f (X n ) , Y n ) , and the equivocation rate E is given by 

£ = -H(Y n \J), J ej. (4) 

n 

Definition 1: The rate-distortion-equivocation tuple (i?, Di,D 2 , E) is achievable for the above 
source network if there exists an (n, M, Di + e, D 2 + e, E — e) code with M < 2 n ^ + ^ for n suffi- 
ciently large. Let 1Z denote the set of all achievable (R, D 1: D 2 , E) tuples, R (D 1: D 2 , E) denote 
the minimal achievable rate R, and T (Di,D 2 ) denote the maximal achievable equivocation E 
such that 

R(D 1 ,D 2 ,E)= min R, and (5) 
r (Di, D 2 ) = max E. (6) 

(iJ,Di,D 2 ,-B)G7e,Vi?>0 

Remark 1: T (Z?i, D 2 ) is the maximal privacy achievable about F™ at decoder 1 and R (D 1: D 2 , E) 
is the minimal rate required to guarantee a distortion pair (D 1: D 2 ) and an equivocation E. 
R (Di, D 2 , T(D 1 , D 2 )) is the minimal rate achieving the maximal equivocation for a distortion 
pair (Di,D 2 ) . 
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III. Related Observations 

In the context of lossless communications, [16] studies a problem of losslessly communicating 
a bivariate source (X, Y) to a single decoder via two encoders, one with access to the X n 
sequences and the other with access to the Y n sequences. A special case of this problem is one 
in which the decoder has perfect access to Y n for which a minimal rate of Rx > H(X\Y) is 
needed [16] and it this problem (which leads to a corner point in the Slepian-Wolf region) that 
we address below. 

On the other hand, [17] studies the problem of lossily communicating a part X of a bivariate 
source (X, Y) subject to a fidelity criterion to a single decoder which has access to Y and proves 
that a minimum rate of R(D) > min (I(X; U) — I(Y; U)) where the minimization is over all 
distributions p (u\x) and deterministic functions g such that X — g(U, Y) and E 
D. 

In both of the abovementioned problems, the coding index communicated is chosen with 
knowledge of the decoder side information. In the lemmas that follow we prove that in both 
cases the optimal encoding is such that the coding index is asymptotically independent of the 
side information Y n at the decoder. 



d [X,X 



< 



A. Slepian-Wolf Coding Coding: Independence of Bin Index and Side Information 

Lemma 1: For a bivariate source (X, Y) where X n is encoded via the encoding function fsw '■ 
X n — > J e {1, . . . , Mj} while Y n is available only at the decoder, we have Hindoo H (Y n \ J)/ n - 
H (Y) , i.e., lim^oo / (Y n ; J)/n ->■ 0. 

Proof: Let Ta {n, e) denote the set of strongly typical A sequences of length n. We define 
a binary random variable p as follows: 

p{x n ,y n ) = I V ,T V ! (7) 
I 1, otherwise. 

From the Slepian-Wolf encoding, since a typical sequence x n is assigned a bin (index) j at 
random, we have that 

Pr (J = j\X n = x n ET x (n, e)) = ±- (8) 

Mj 

and 

Pr (J = 3 \p=l) = J2 Pr (x n , J = j\p = 1) G ((1 - e) /Mj, l/Mj) (9) 
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where we have used the fact that for a typical set Pr (Txy ( n , e )) > (1 — e) [18, chap. 2]. 
The conditional equivocation H (Y n \J) can be lower bounded as 

H(Y n \J) > H(Y n \J,fi) (10) 

= Pr (/i = 0)H(Y n \J,/i = 0) + Pr(/i = 1) H (Y n \J, /x = 1) 

> Pr (/x = l)H (Y n \J,/j = 1) (11) 

= Pr (/i = 1) J2j Pr (j'lM = 1) # (Y n \j, H = 1) (12) 

where (flOl ) follows from the fact that conditioning does not increase entropy, and (fTTT) from the 
fact that the entropy is non-negative. The probability Pi(y n \j ) /x = 1) can be written as 

Pr(^|j,/x = l) 

= £Pr(?/\;r"|j, /i = 1) (13a) 

= E Pr Mi, // = 1) Pr (y n \x n , j, /x = 1) (13b) 

<2"'-E Pr(x1/f J 1)/Mj Prtey,^l) (13d) 

= 2 ne '^ Pr (x n \fi = 1) Pr (y n |x n , /x = 1) (13e) 

= 2 ne 'Pr (y n |/x = 1) (13f) 

< 2 -^( y )-") (13g) 

where (I13bl) follows from © and the fact that F n — X n — J forms a Markov chain (by 
construction), and (I13dl) follows from ©. Expanding H (Y n \j, /x = 1), we have 

jt (y» |i, /x = i) = j> (y n \j, m = i) log ^— ^ 7t d4) 

>Ep(l/ n |j,/i=l)log2"W y )- £ ") (15) 

= n(H(Y)-e")j:p(y n \j, l x = l) (16) 

>n(l-e)(#(r)-e") (17) 
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where (fl"5l) results from the upper bound on Pr (y n \j, [i — 1) in (|13g[) and (fT71) from the fact that 



for a typical set Pr (Txy (n,e)) > (1 — e) [18, chap. 2]. Thus, the equivocation H (Y n \J) can 
be lower bounded as 

H (Y n \J) > Pr (/i = 1) £ . Pr = 1) (1 - e) n (# (Y) - e") (18) 
>n(l-e) 3 (iJ(r)-e") (19) 

where we have used © and the fact that for a typical set Pr (Txy (n, e)) > (1 — e) [18, chap. 
2]. The proof concludes by observing that H (Y n ) > H (Y n \J) and e — > 0, e" — > as n — > oo. 

■ 

Remark 2: Lemma Q] captures the intuition that it suffices to encode only that part of X n that 
is independent of the decoder side-information Y n . 

Remark 3: The proof of Lemma \T\ does not depend on the precise bound on the total number, 
Mj, of encoding indices, i.e., it holds for all choices of Mj. In fact, the bound on Mj is a 
consequence of the decoding requirements. 

B. Wyner-Ziv Coding: Independence of Bin Index and Side Information 

Lemma 2: For a bivariate source (X, Y) where X n is encoded via the encoding function fwz '■ 
X n — > J e {1, . . . , Mj} while Y n is available only at the decoder, we have lim^oc H (Y n \ J)/ n - 
H (Y) , i.e., lim^ / (Y n ; J)/n -> 0. 

Proof: Let Ta (n, e) denote the set of strongly typical A sequences of length n. We define 
a binary random variable /i as follows: 

I 0, (u n , y n ) & T UY (n, e) ; 
H(u n ,y n ) = I K ,T K ' (20) 

i 1, otherwise. 

From the Wyner-Ziv encoding, for a given x n , first a sequence u" that is jointly typical with x n 
is chosen where the n symbols of u n are generated independently according to pu (•) (computed 
from pxu ("))■ The resulting sequence u n is assigned a bin (index) j at random such that we 
have 

Pr (J = j\U n = u n G % (n, e)) = (21) 

Mj 

and 

Pr (J — Pr (u n , J = = 1) G ((1 - e) /Mj, l/Mj) (22) 
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where we have used the fact that the probability of the typical set Tuy ( n i e ) > (1 — e ) [18, 
chap. 2] and using (1 — e) /Mj = 2~ n€ ' /Mj for a given n. 

The conditional equivocation H (Y n \J) can be lower bounded as 

H(Y n \J) > H(Y n \J,/j) (23) 

= Pr (/i = 0)H(Y n \J,n = 0) + Pr(/i = 1) H (Y n \J, /j, = 1) 

> Pr (fi = l)H (Y n \J,/2 = 1) (24) 

= Pr (/i = 1) J2j Pr =l)H (Y n \j, n = 1) (25) 

where (flOl) follows from the fact that conditioning reduces entropy, and (fTTI) from the fact that 
the entropy is non-negative. The probability Pr(y n \j, fj, = 1) can be written as 

Pr(y n |^ = l) (26a) 

= J2Pr(y n ,u n \j, f i=l) (26b) 

= £ Pr /i = 1) Pr (j/»V, j, /i = 1) (26c) 

= E Pr (« n |j, /i = 1) Pr (</> n , /i = 1) (26d) 

= E p'"; l " = 1 1 ' ^PryK„=l) (26e) 

u n Pr(j|// = 1) Mj 

< £ Pr (w n |/i = 1) 2 ne ' Pr (y n |u n , fi = 1) (26f) 
= £Pr(</» = l)2" e ' (26g ) 

= Pr(y n |/i = l)2 ne ' (26h) 

< 2 - n ^( y )- £ ") (26i) 

where (|26d| ) follows from (|2T1) and the fact that y n — U n — J forms a Markov chain (by 
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construction) and (I26fl) follows from (|22~1) . Expanding H (Y n \j, [i = 1), we have 

H (Y n \j, v=l) = ZP (y n \3i A* = 1) log t^t^ tt (27) 

>Ep(2/ n |j,/i = l)log2" Wy) - £,) (28) 

= n{H{Y)-e>)Yj>{y n \j,l* = l) (29) 

> n (1 - e) (if (K) - e') (30) 

where (fl"5l) results from the upper bound on Pr (y n |j, jj = 1) in (I26il) and (fTTT) from the fact that 
for a typical set T X y (n,e) > (1 — e) [18, chap. 2]. Thus, the equivocation H (Y n \J) can be 
lower bounded as 

H (Y n \J) > Pr (/x = 1) £. Pr = 1) (1 - e) n (# (F) - e') (31) 

>n(l-e) 3 (H(Y)-e') (32) 

where we have used the fact that for a typical set Tuy {n>, e) > (1 — e) [18, chap. 2]. The proof 
concludes by observing that H (Y n ) > H (Y n \ J) and e' — ^ 0, e" — > as n — > oo. ■ 
We will now use Lemmas CD and [2] to demonstrate the optimality of the Heegard-Berger and 
Kaspi encoding for the uninformed and informed source models respectively. 

C. Uninformed Encoder with Side Information Privacy 

We first consider the source network in which the encoder does not have side information 
and derive the set of all feasible rate-distortion-equivocation (RDE) pairs. The resulting problem 
may be viewed as the Heegard-Berger problem with an additional privacy constraint at decoder 
1. Our result demonstrates that the optimal coding scheme is the same as the Heegard-Berger 
problem without a privacy constraint. The proof makes use of the independence of the Wyner-Ziv 
binning index from the side information Y n in tightly bounding the achievable equivocation. We 
briefly sketch the proof here; the detailed proof can be found in the appendix. 

1) Rate-Distortion-Equivocation (R, Di, D 2 , E) Tuples: 

Definition 2: Let T u (Di, D 2 ) and Ru (Di, D 2 , E) be two functions defined as 

T U (D 1 ,D 2 )= max H{Y\W X ) (33) 

Vu(Di,D 2 ,E) 

Ru (D 1 , D 2 , E) = min I(X; W x ) + I(X; W 2 \W 1 Y) (34) 
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such that 

Ku = {(R, D u D 2 , E) : D x > 0, D 2 > 0, < E < Tu (£>i, D 2 ),R> R v (D h D 2 , E)} (35) 

where the subscript U denotes the uninformed case, V\j (D 1 , D 2 , E) is the set of all p (x, y)p{w\, w 2 \x) 
that satisfy © and ©, Y - X - (Wi, W 2 ) is a Markov chain, and |Wi| = \X\ + 2, \W 2 \ = 
(\X\ + l) 2 . 

Lemma 3: Tu (Di, D 2 ) is a non-decreasing, concave function of (D 1 , D 2 ) (i.e., for all Di > 0, 
1 = 1,2). 

Lemma [3] follows from the concavity properties of the (conditional) entropy function as a 
function of the underlying distribution, and therefore, of the distortion. 

Theorem 1: For a bivariate source (X,Y) where only X n is available at the source, and Y n 
is available at decoder 2 but not at decoder 1, we have 

K = Ku, T(D U D 2 ) = T U (D U D 2 ), and R(D X ,D 2 ,E) = R v (D l7 D 2 ,E) . (36) 
Proof sketch: Converse: A lower bound on R(Di, D 2 , E) is the same as that in [1] and 
involves the introduction of two auxiliary variables Wx t i = (J, Y 1 ^ 1 ) and W 2ti = 
Using this definition of Wi^, one can expand the equivocation definition in © to show that 
T(D 1 ,D 2 )<H(Y\W 1 ). 

Achievable scheme: The achievable scheme begins with a rate-distortion code for decoder 1 
by mapping an observed x n sequence to one of a set of 2 nI{ - x ' ,Wl " > sequences, denoted w" (ji), 
subject to typicality requirements. For this choice of w™ (ji), a second code for decoder 2 results 
from choosing a conditionally typical sequence out of a set of 2 nI ( x '< W2 \ Wl *> w% sequences, denoted 
by w% {j 2 \ji), and binning the resulting sequence into one of 2 n ^ Xv ' w ^ Wl> >-^ Y ' w ^ Wl ^ bins, 
denoted by b (j 2 ), chosen uniformly. The pair (ji, b (j 2 )) is revealed to the decoders. We show in 
the appendix that this scheme achieves an equivocation of H (Y\Wi) asymptotically; the crux of 
our proof relies on the fact that the binning index B (J 2 ) is conditionally independent of (XWi) 
conditioned on W 2 , i.e., the random variables are related via the Markov chain relationship 
Y -{XW 1 )-W 2 -B(J 2 ). 

Remark 4: An intuitive way to interpret the equivocation arises from the following decom- 
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position: 

-H{Y n \J u B{J u J 2 )) = -H{Y n \.h) (37a) 
n n 

--I{Y n -B{J 2 )\J 1 ) (37b) 
n 

= -H(Y n \W?(J 1 )) (37c) 

n 

- -I (Y n ; B (J 2 ) \Wi ( Ji)) . 
n 

The first term in (I37cl) is approximately equal to i?(Y|Wi) while the second term, which in 
the limit goes to 0, follows from a conditional version of Lemma [2] and the fact that Y — X — 
(WiW 2 ) — B (J 2 ) forms a Markov chain. 

D. Informed Encoder with Side Information Privacy 

We now consider the source network in which the encoder has access to the side information 
Y n and derive the set of all feasible rate-distortion-equivocation tuples. The resulting problem 
may be viewed as the Kaspi problem with an additional privacy constraint about Y n at decoder 1. 
Our results below demonstrate that the Kaspi coding scheme achieves the set of all rate-distortion- 
equivocation tuples. However, for a given (D x , D 2 , E) pair, the minimal rate R(D t , D 2 , E) will 
in general be different from the R(D 1 , D 2 ) for the original Kaspi problem. 

Our proof includes a two-step achievable scheme involving binning for the conditional rate- 
distortion function for which we show that the bin index is independent of the side information 
Y n . Our converse is a minor modification of the converse in [2] and involves two auxiliary 
random variables. We briefly sketch the proof here; the details are relegated to the appendix. 

I) Rate-Distortion-Equivocation (R, D\, D 2 , E) Tuples: 

Definition 3: Let T I (D 1 ,D 2 ) and Rj (Di,D 2 ,E) be two functions defined as 

TAD^D^^ max HiYlWA, and (38) 

Vi(Di,D 2 ,E) 

R I (D 1 ,D 2 ,E)= min I(XY; Wi) + I(X; W 2 \WiY) (39) 

Vi(Dx,D 2 ,E) 

such that 

Hi = {(R, D x , D 2 , E) : D x > 0, D 2 > 0, < E < Tj (D u D 2 ),R> R T (D u D 2 , E)} (40) 
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where Vi {D\, D 2 , E) is the set of all p (x, y)p(wi, w 2 \x, y) that satisfy © and © and \W\ \ = 
\X\+2, \W 2 \ = (\X\ + 1) 2 . 

Remark 5: The cardinality bounds on Wi and W 2 can be obtained analogously to the argu- 
ments in [1, p. 730]. 

Lemma 4: Rj D 2 , E) is a convex function of (Di, D 2 ,E). 

Theorem 2: For a two-source (X, F) where X n is available at the source, and Y n is available 
at the source and at decoder 2 but not at decoder 1, we have 

n = Hi, r(D 1 ,D 2 ) = r I (D 1 ,D 2 ), md r(d 1 ,d 2 ,e) = r i (d 1 ,d 2 ,e). (4i) 

Proof sketch: Converse: A lower bound on R D 2 , £7) can be obtained analogously to the 
bounds in [2] with the introduction of two auxiliary variables W\i = (J, Y 1 ^ 1 ) and W 2:i = 
(X^Y-^i) . Using this definition of Wi^, one can expand the equivocation definition in © to 
obtain r(£>i,D 2 ) < H{Y\W X ). 

Achievable scheme: The achievable scheme begins with a rate-distortion code for decoder 1 
by mapping an observed (x n ,y n ) sequence to one of a set of 2 nI ( XY ' Wl ^ u>™ sequences, denoted 
by W™ (ji), subject to typicality requirements. A second rate-distortion code for decoder 2 results 
from mapping (x n ,y n ,w±) to one of a set of 2 nI ( XYWl ' W2 *> w 2 sequences, denoted by w 2 (j 2 ), 
and binning the resulting sequence into one of 2 n( - I( - XYWl ' w ^- I( - YWl ' W2 ^ bins, denoted by b (j 2 ), 
chosen uniformly. The pair (ji,b(j 2 )) is revealed to the decoders. In the appendix it is shown 
that this scheme achieves an equivocation of H (Y\Wi); the crux of the proof relies on the fact 
that the binning index B ( J 2 ) is conditionally independent of (XYWi) conditioned on W 2 . 

Remark 6: An intuitive way to interpret the equivocation arises from the same decomposition 
as in (|37l) where the first term in (I37cl) is approximately equal to H (Y\Wi) while the second 
term, which in the limit goes to 0, follows from a conditional version of Lemma [2l Note that, 
in contrast to the uninformed case, the distribution here is such that (XY) — (WiW 2 ) — B (J 2 ) 
forms a Markov chain. 

IV. Results for a Binary Source with Erased Side Information 
We consider the following pair of correlated sources. X is binary and uniform, and 




X, w.p. (1 - p) 
E, w.p. p, 
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and we consider the Hamming distortion metric, i.e., d(x, Xj — X © X for both decoders and for 
both the informed and uninformed cases. 



A. Uninformed Case 

We are interested in the rate-distortion-equivocation tradeoff, given as, 

R > /(X;Wi) + I(X;W 2 \Y,W 1 ), and (42) 
E < H(Y\Wi) (43) 

where the rate and equivocation computation is over all random variables (W%, W 2 ) that satisfy 
the Markov chain relationship (Wi, Wjj) — X — Y and for which there exist functions /1 (•) and 
fi (;•>') satisfying 

E[d(X,f 1 (W 1 ))]<D 1 , and (44) 
E[d(XJ 2 (W u W 2 ,Y))}<D 2 . (45) 

Let h (a) denote the binary entropy function defined for a E [0, 1]. The (_D 1; D 2 ) region for this 
case is partitioned into four regimes as shown in Fig. [21 
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1.6 - 




D 1 



Fig. 3. Illustration of the rate-equivocation tradeoff for p — 0.25. 



The rate-distortion-equivocation tradeoff is given as follows: 

f 

0; 

p(l-h(D 2 /p)); 

l-MA); 

p(l - h(D 2 /p)) + ( 



R(D 1 ,D 2 ) 



if (D u D 2 ) e A, 
if (D 1 ,D 2 )e£ 2 , 

if (£>i, £> 2 ) e £ 3 , 
if (D 1 ,D 2 )eU 



and 



rpi,z> 2 ) 



+ if£>i<l/2, 

+ (1 — p); otherwise. 

In Figure [3l we have plotted R(D 1 , D 2 ) and r(Di, D 2 ) for the cases in which D 2 = p/2 and 
D 2 =p/8, and D x G [0,1/2]. 

Remark 7: This example shows that the equivocation does not depend on the distortion 
achieved by the decoder 2 which has access to side-information Y, but rather depends only 
on the distortion achieved by the uninformed decoder 1. 

1) Upper bound on V{D\, D 2 ): For any Di > 1/2, we use the trivial upper bound 

F(D 1 ,D 2 )<H(Y\W 1 )<H(Y) (46) 
= h{p) + 1 - p. (47) 
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For any D x < 1/2, we use the following: 

r(Di, D 2 ) < H(Y\ Wx) (48a) 

= H(Y,X\W!) - H{X\Y ) W 1 ) (48b) 

= H(X\W l ) + H{Y\X)-H(X\Y,W 1 ) (48c) 

= H(X\Wi) + H(Y\X) - pH{X\W x ) (48d) 

= H(Y\X) + (1 - p)H(X\W 1 ) (48e) 

= H(Y\X) + (1 -p)H(X\W 1 ,X 1 ) (48f) 

<H(Y\X) + (l~p)H(X\X 1 ) (48g) 

< If (K |X) + (1 - p)ff (X © Xi) (48h) 

= if + (1 - p)/i(P(X ^ 10) (48i) 

</ l (p) + (l-p)/ l (L> 1 ) (48j) 

where (I48dl) follows from a direct verification that if (X|Y, Wi) = pH (X\W\) if X is uniform 
and Y is an erased version of X and W\ — X — Y forms a Markov chain. 
1) Upper bound on T(Di,D 2 ): 

• If (Z?i, D2) G A, we use the lower bound R(D\, D 2 ) > 0. 

. If (Di,D 2 ) G £ 2 , we use the lower bound R{D U D 2 ) > R% ] Z [D 2 ) [19]. 
. If (£>!, £> 2 ) G £ 3 , we use the lower bound R(D U D 2 )>1- h(D x ). 

• If (Di,D 2 ) G £4, we show that 

R(Di, D 2 ) > p{\ - h(D 2 /p)) + (1 -p){l - h{D x )). (49) 

Consider an arbitrary (W x , W 2 ) such that (W\, W 2 ) — > X — > F is a Markov chain and there 
exist functions f x and / 2 : 

X = h{W x ), and X 2 = f 2 (W 1 ,W 2 ,Y), 

such that 

Pr(X^X J )<D J , j = 1,2. 



June 13, 2011 



DRAFT 



17 

Now consider the following sequence of equalities: 

/(X; W x ) + /(X; W 2 \Y, W x ) = H(X) - H(X\W X ) + H{X\Y, W x ) - H(X\Y, W u W 2 ) 

= H(X) - /(X; Y\W X ) - H(X\Y, W h W 2 ) 
= H(X) - H{Y\W X ) + H(Y\X, - H(X\Y, W u W 2 ) 
= H(X) + H(Y\X) - H(Y\W X ) - H{X\Y, W u W 2 ). (50a) 
Consider the following term appearing in (150al) : 

H(Y\Wx) = H(Y, X\Wi) -H{X\Y,W X ) (51a) 

= H{Y\X) + H(X\W 1 ) -H(X\Y,W 1 ) (51b) 

= H(Y\X) + (1 - p)H(X\W 1 ) (51c) 

= H(Y\X) + (1 -p)H(X\W 1 ,X 1 ) (5 Id) 

<H(Y\X) + (l-p)H(X\X 1 ) (51e) 

<H(Y\X) + (l-p)H(X®X 1 ) (51f) 

<H(Y\X) + {l-p)h{D x ). (51g) 

We also have 

D 2 > Pr(X ± X 2 ) (52a) 

= Pr(F = E) Pr(X ^ X 2 \Y = E) + Pr(F ^ E) Pr(X ^ X 2 \Y ^ E) (52b) 

> Pr(F = E) Pr(X ^ X 2 \Y = E) (52c) 

= pPr(X^X 2 |F = £) (52d) 
which implies that 

Yt(X^X 2 \Y = E)< — <\. (53) 

p I 
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Now consider the following sequence of inequalities for the last term in (150al) : 



H{X\Y, W 1} W 2 ) = H(X\Y, W l7 W 2 , X 2 ) 



(54a) 



<H(X\Y,X 2 ) 



(54b) 



pH(X\Y = E,X 2 ) 



(54c) 



<pH(X@X 2 \Y = E) 



(54d) 



ph(P(X ^ X 2 \Y = E)) 



(54e) 



< ph(D 2 /p) 



(54f) 



where (|54f|) follows from (|53l ). Using ( 51g[ ) and (|54f|) , we can lower bound <150aL to arrive at 



R{D 1} D 2 ) > p(l - /i(D 2 /p)) + (1 - p)(l - 



5) Coding Scheme: 

. If {D X ,D 2 ) e d, the (i?,T) tradeoff is trivial. 

• If (Di,D 2 ) E C 2 , we use the following coding scheme: 
In this regime, we have Di > 1/2, hence the encoder sets W\ = 4>, and sends only one 
description W 2 = X © N, where N ~ Ber(D 2 /p) and is independent of X. It can be 
verified that /(X; W 2 |^) = p(l — h(D 2 /p)). Decoder 2 estimates X by X 2 as follows: 



Therefore the achievable distortion at decoder 2 is (1 — p)0 +p(D 2 /p) = D 2 . 
• If (Di,D 2 ) G £3, we use the following coding scheme: 
The encoder sets W 2 = (ft, and sends only one description W\ = X®N, where N ~ Ber(Di) 
and N is independent of X. It can be verified that I(X; Wx) = 1 — h(Di). Decoder 1 
estimates X as X\ = W\ which leads to distortion of D\. Decoder 2 estimates X by X 2 
as follows: 



Therefore the achievable distortion at decoder 2 is (1 — p)0 + p(Di) = pD\. Hence, as long 
as D 2 > pDi, the fidelity requirement of decoder 2 is satisfied. 
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If (D 1 ,D 2 ) G £4, we use the following coding scheme: 

We select W 2 = X © N 2 , and W 1 = W 2 ® N u where N 2 ~ Ber(D 2 /p), and JVi ~ Ber(a), 
where a = (Dx — D 2 /p)/(l — 2D 2 /p), and the random variables iVi and iV 2 are independent 
of each other and are also independent of X. At the uninformed decoder, the estimate is 
created as X\ = W\, so that the desired distortion Dx is achieved. 
At the decoder with side-information Y, the estimate X 2 is created as follows: 

Y; if Y ^ E- 

W 2 ; if Y = E. 

Therefore the achievable distortion at this decoder is (1 — p)0 + p(D 2 /p) = D 2 . It is 
straightforward to check that the rate required by this scheme matches the stated lower 
bound on R(D U D 2 ), and r(D u D 2 ) = #(T|Wi) = h(p) + (l-p)h{D x ). This completes 
the proof of the achievable part. 

B. Informed Encoder 

For this case, the rate-distortion-equivocation tradeoff is given as 

R > I(X, Y] W x ) + /(X; W 2 \W 1 ,Y), and (55) 

E < H{Y\W X ) (56) 

where the joint distribution of (Wi, W 2 ) with (X, Y) can be arbitrary. 

As in the previous section, we partition the space of admissible (D x , D 2 ) distortion pairs. For 
simplicity, we denote these partitions as follows: 

Q x = {(A, D 2 ) : D 1 > 1/2, D 2 > p/2}, (57) 

G 2 = {(£>!, D 2 ) : D x > 1/2, D 2 < p/2}, (58) 

G 3 = {(Di, D 2 ) :D 1 >D 2 + (1- p)/2, D 2 < p/2}, (59) 

Ga = {(Dx, D 2 ) : Dx < 1/2, D 2 > Dx}, and (60) 

£ 5 = {{Dx, D 2 ) :Dx<D 2 + {l- p)/2, D 2 < Dx}. (61) 

These partitions are illustrated in Figure HI 

We provide a partial characterization the optimal (R, E) tradeoff as a function of (Dx, D 2 ). In 
particular, we establish the tight characterization of (R, E) pairs for all values of (Dx, D 2 ) with 
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D 2 
1/2 

p/2 



(0,0) (l-p)/2 1/2 1 

Fig. 4. Partition of (D\,D2) region: informed encoder case. 

the exception of when (Di,D 2 ) G </5. This characterization reveals the benefit of the encoder 
side-information. It shows that in the presence of encoder side-information, there can be several 
(R, E) operating points relative to the case in which the encoder does not have side-information. 

(a) (Z?i, D 2 ) E G\ : In this case the (R, T) region is trivial since both the decoders can satisfy 
their distortion constraints which also yields the maximum equivocation, i.e., we have 

R(D 1 ,D 2 ) = 0, and (62) 
T(D 1 ,D 2 ) = h(p) + l-p (63) 

(b) (Di, D 2 ) € Q 2 : In this case, we use the proof as in the uninformed case for the partition 
C 2 to show that 

R(D U D 2 ) = p(l - h(D 2 /p)), and (64) 

T(D 1 ,D 2 ) = h(p) + l-p. (65) 

(c) (D 1 ,D 2 ) G Qz : The (R,T) tradeoff for this case is given as follows: 

R(D U D 2 ) = p(l - h(D 2 /p)), and (66) 
T{D l ,D 2 ) = h(p) + l-p. (67) 
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0,0 




Wi 







1,1 



1 



Fig. 5. Illustration of p(wi\x, y) when D x > D 2 + (1 - p)/2 and L> 2 £ [0,p/2]. 

This case differs from the uninformed encoder case in the sense that for the same rate, we 
can achieve the maximum equivocation and a non-trivial distortion for decoder 1. Since R > 
Rx\y(D 2 ) = R^ Z (D 2 ), and T < H(Y), the converse proof is straightforward. The interesting 
aspect of this regime is the coding scheme, which utilizes the side information at the encoder 
in a non-trivial manner. To achieve this tradeoff, we set W 2 = 0, and send only one description 
W\ to both the decoders. The conditional distribution p(wi\x,y) that is used to generate the 
codewords is illustrated in Figure \5\ . 

Hence the rate for this scheme is given by 



R > I{X,Y;Wi) 



(68) 



H(W 1 )-H(Wi\X,Y) 



(69) 



l-H(Wi\X,Y) 



(70) 



l-(l-p)-ph{D 2 /p) 



(71) 



p(l - h(D 2 /p)) 



(72) 
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and the equivocation is given as 

r = #(y|Wi) (73) 

= H{Y)-I{Y-W l ) (74) 
= H(Y)-H(W 1 ) + H(W 1 \Y) (75) 
= H{Y)-1 + H(W 1 \Y) (76) 
= if (y) - 1 + (1 - p)H(W l \Y = X)+ pHiW^Y X) (77) 
= H(Y)-l + (l-p)+p (78) 
= #Q0- (79) 
Decoder 2 forms its estimate as follows: 

Y ifY^E; 
W x if Y = E, 

which yields a distortion of _D 2 at decoder 2. Decoder 1 forms its estimate as 

Xx = W 1 



Xo = 



which yields 



Therefore, as long as 



F(X 1 ^X) = D 2 + [[ " l>] 



Di > D 2 + 



2 

(1-p) 



2 ' 

this scheme achieves the optimal (R, T) tradeoff. 

We now informally describe the intuition behind this coding scheme: since the encoder has 
access to side-information Y, it uses the fact that whenever Y = X, no additional rate is required 
to satisfy the requirement of decoder 2, i.e., for (1 —p) -fraction of time it is guaranteed to exactly 
recover X. However, this yields a distortion of (1 —p)/2 at decoder 1 (since decoder 1 does 
not have access to Y). In the remaining p-fraction of time, the encoder describes X with a 
distortion D 2 /p, which contributes to a distortion of D 2 at both the decoders. To summarize, 
the net distortion at decoder 2 is D 2 , whereas the distortion at decoder 1 is lowered from 1/2 to 
(1 — p)/2 + D 2 . Furthermore, by construction, W\ is independent of Y, i.e., H{Y\W\) = H(Y), 
which results in the maximal equivocation at decoder 1 . 
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X, 



(d) (Di, D 2 ) G Q4 : For this case, the (R, E) tradeoff is given as the set of (R, E) pairs 

R>l-(l-p)h ( ^j ~ P h (a), and (80) 

E < h(p) + (1 - p)h (2^^) , (81) 

where the parameter a belongs to the range a G [0, Di/p]. 

We now describe the coding scheme that achieves this region: we set W 2 = 4>, and send one 
description W\ at a rate I(X,Y;Wi). The conditional distribution p(wi\x, y) that is used to 
generate the codewords is illustrated in Figure [6j The parameters (a, (5) that describe this 
distribution are chosen such that 

D l > F(X ^ Wi) (82) 
>(l-p)P+pa, (83) 
so that /3 < {Di — pa) /(I —p). At decoder 2, the estimate X 2 is created as 

Y; if Y + E- 
Wr, ]fY = E, 

which yields a distortion of pa. Since a E [0, Di/p], the worst case distortion for decoder 2 for a 
fixed Di is p(Di/p) = D x . Hence, as long as D 2 > D lf we can satisfy the fidelity requirements 
at both decoders. By direct calculations, it can be shown that the resulting (R, E) tradeoff is as 
stated above. 

Compared to all the previous cases, the proof of optimality of the above coding scheme is 
non-trivial and is relegated to the appendix. 

We remark here that in this regime, the tradeoff between rate and privacy can be observed 
in a precise manner. First, note that the choice a = D x yields the (R, E) operating point as in 
the uninformed encoder case. Next, when a decreases from D\ to 0, the equivocation increases, 
albeit at the cost of a higher rate. This phenomenon does not occur in the case in which the 
encoder does not have side information. 

Finally, when a is in the range (Z?i, Di/p], we obtain a lower equivocation by increasing the 
rate. This phenomenon appears counterintuitive and can be explained as follows: this range of a 
corresponds to a coding scheme in which we give more weight to the side-information Y when 
describing X to decoder I. Such a coding scheme can be regarded as the solution to the problem 
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Fig. 6. Illustration of p(wi|x, y) when Di < 1/2, D2 > Di. 

in which the encoder is interested in revealing Y to decoder 1, while simultaneously satisfying 
the fidelity requirement for X at decoder 1. While it is a feasible solution to the problem, it 
may not be a desirable coding scheme when the privacy of Y at decoder is of primary concern, 
and thus, there exists a set of rate-equivocation operating points that one can choose from. In 
Figure [7J we show the (R, E) achievable tradeoff when p = 0.4 and D\ = 0.2. 
(d) (Di,D 2 ) € G5 '■ For this case, the following (R, E) pairs are achievable: 



where a is such that a G [0,D 2 /p]. The coding scheme that achieves this tradeoff is similar 
to the one used when (D 1 ,D 2 ) £ G4, with the exception that the range of a is different. The 
question of optimality of tradeoff for this regime is still unresolved. 



We have determined the rate-distortion-equivocation region for a source coding problem with 
two decoders, in which only one of the decoders has correlated side information and it is 
desired to keep this side information private from the uninformed decoder. We have studied 




(84) 



(85) 



V. Concluding Remarks 
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Fig. 7. Illustration of the rate-equivocation tradeoff for p — 0.4, Di = 0.2 with an informed encoder. 

two cases of this problem depending on the availability of side information at the encoder. We 
have proved that the Heegard-Berger and the Kaspi coding schemes are optimal even with an 
additional privacy constraint for the uninformed and the informed encoder cases, respectively. 
We have illustrated our results for a binary symmetric source with erasure side information and 
Hamming distortion which clearly highlight the difference between the informed and uninformed 
cases and the advantages of encoder side information for enhancing privacy. Future work includes 
generalization to multiple decoders as well as to continuously distributed sources. 



A. Proof of Theorem [7] 

Converse: The lower bound on R (D\, D 2 , E) follow directly from the converse for the 
Heegard-Berger problem and is omitted here in the interest of space. We now upper bound 



Appendix 
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the maximal achievable equivocation as 



H(Y n \J) = J2-H (Yi\Y l ~ 



(86a) 



n i=1 n 




(86b) 



<rv (Di,d 2 ) 



(86c) 



where (I86bl) follows from defining Wi t i = (J, F^ 1 ) (see [1, sec. IV]) and (I86cl) follows from 
the definition of IV (Z? 1( D 2 ) in (|33l) and its concavity property from Lemma [3j 

Achiev ability: We briefly summarize the Heegard-Berger coding scheme [1]. Fix p (wi, W2\x). 
First generate Mi = 2 n ^ Wl ' x ^ +e \ (ji) sequences, ji = 1,2, ...,M±, independently and 
identically distributed (i.i.d.) according to p(w{). For every (ji) sequence, generate M 2 = 
2n(i(w 2 -,x\w 1 )+e) W£(j 2 \ji) sequences i.i.d. according to p{w 2 \wi{j\)). Bin the resulting W 2 
sequences into S bins (analogously to the Wyner-Ziv binning), chosen at random where S = 
2n{i(X]W 2 \Wi)-i(Y;W 2 \Wi)+e) ^ anc j m( j ex these bins as b (j 2 ). Upon observing a source sequence x n , 
the encoder searches for a W[ l (ji) sequence such that (x n ,w™ (ji)) G Txwi e ) ( tne choice 
of Mi ensures that there exists at least one such Next, the encoder searches for a u> 2 (j 2 \j\) 
such that (ji) , u> 2 (j'2 |ji)) 6 7xWiW 2 ( n 5 e ) ( me choice of M 2 ensures that there exists at 

least one such j 2 ). The encoder sends 6 (j 2 )) where 6 (j 2 ) is the bin index of the w 2 (j 2 \ji) 
sequence. Thus, we have that (XWi) — W 2 — B forms a Markov chain and 



With fx as defined in © for the typical set TxywiWv an d J = (Ji, B (J 2 )), the achievable 
equivocation can be lower bounded as 



Pr (B = b (j 2 ) | (x n , < (ji) , w n 2 (j 2 |ji)) G Txwm (n, e)) 



Pr(5 = &(j 2 )K(j 2 |ji) GT W2 (n,e 



)) = 1/5. 



(87) 



n 

= ^H{Y n \W?{J l ) ) B{J 2 )^) 

> Pr (/i = 1) iff {Y n \ W? ( Ji) , S ( J 2 ) ,/i = 1) . 



(88b) 



(88c) 



(88a) 
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The probability Pr (y n \w™ (ji) , b (j 2 ) , /x = 1) for all ji, j 2 , and y n can be written as 

E Pr(y n ) j a ,x>rOi),6(7a)^ = l) 

= E Pr(x",j 2 K(j 1 ),6(j 2 ), / i=l)Pr( 2/ "|x",/i=l) (89a) 

<2 ne ' E Pr(x m ,j 2 K(j 1 ),/i=l)Pr( 2/ "|x n , / x=l) (89d) 

(x n J 2 ) 

= 2^' £ Pr(x n ,i JlS / n K0'i)^ = l) (89e) 

= 2-'Pr( 2 /"K(j 1 ),/i = l) (89f) 

where (I89al) follows from the fact that Y — X — (Wi, W 2 ) forms a Markov chain and (|89dl) is 
obtained by expanding Pr (jj) , b (j 2 ) |)U = 1) as follows: 

Pr« 0*i).6 0*2) |A* = 1) 

= Pr « |/i = 1) E Pr (6 (j'2) , ^ 2 n (ji) K (ji) , /x = 1) (90a) 

= Pr « (iO |/i = 1) E Pr « (j'O K &) . A* = 1) 4 (90b) 

> Pr « (Ji) |/i = 1) (90c) 

9— ne' 

= PrK( Jl )|/i=l)— (90d) 

where (|90b| ) follows from the fact that Wi — W 2 — B forms a Markov chain and (1871) . while 
( I90cb follows the fact that for a typical set Pr {TwiW 2 i n i e )) > (1 ~ e ) [18, chap. 2]. Thus, from 
(1891) we have that 

Pr (y>? (j'O , 6 (j 2 ) , /i = 1) < 2-' Pr (y>? , » = 1) (91) 

< 2 -n(H(Y\W 1 )-e") _ ^ 92 ) 
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From (I88cl) and (l92l ), we then have 

H (Y n \w^ (jO , 6 (j 2 ) , /i = 1) > E Pr (l/>? (ji) , A* = 1) " (# (V | Wi) - e") (93) 

>n(l-e)(fl"(Y|Wi)-e") (94) 

such that 

lif OH J) > Pr (ai = 1) i E Pr « (Ji) , & (J 2 ) |A* = 1) # (Y>l (Ji) , b (h) , fi = 1) 

n n w 1 l,b(j 2 ) 

(95) 

> (l-e) 3 (iy(y|W 1 )-e") (96) 

where we have used the fact that for a typical set Pt(TyWiW 3 ( n > e )) > (1 ~ e ) [18, chap. 2]. 
The proof concludes by observing that H (Y n ) > H (Y n \J) and e — > 0, e" — > as n — > oo. 

5. Proof of Theorem [2] 

Converse: A lower bound on i? (Z?i, L>2, -S) can be obtained as follows. 

nR>H(J) (97a) 
> /(X n F n ; J) (97b) 
= I(X n ; J\Y n ) + I(Y n ; J) (97c) 
= E JX^Y^Y^Y,) -IiX.-X^Y^Y^Y,) +I(Y i ; J, Y^ 1 ) - I(Y i5 Y 1 - 1 )} 

i=l 

= E { JX^Y^Y^IY) +/(Y <; JY^ 1 )} (97d) 

i=l 

= E {/(X; JF 1 - 1 !^) +/(X,;X^ 1 F^ 1 |jr j - 1 F l ) JF' 4 - 1 )} (97e) 

i=l 

where (I97dl) follows from the independence of the pairs (Jfj,Yj) for all i = 1, 2, ...,n. Let 
= (J, y i_1 ) and JY 2 ,i = (X^Y^). With these definitions, (l97el) can be written as 

n 

nR>Z {IiXiYi- W u ) + I(Xi- W 2 , t \W l ,Y l } (98) 
i=i 

n 

> J2R I {D 1>i ,D 2>i ,E i ) (99) 
i=i 

>nR I {D 1 ,D 2 ,E) (100) 
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where (|99| ) follows from Definition [3] with Dii, D 2i , and Ei defined as 

D 1>i = E[d(X i ,g' 1:i (W 1 , i ))] (101a) 
D 2>i = E[d(X i ,g' 2:i (W 2 ,i,W 1 , i ,Y i ))] , and (101b) 
Ei = H(Y t \W hl ) } (101c) 

and (1 1001) follows from the convexity of Ri(Di, D 2 , E) and the definitions of Dk, k — 1, 2, in 
© and the concavity of (FlW 7 ), and hence, of E. We upper bound the maximal achievable 
equivocation as 

1 n 1 

(y n | J) = J2-H (Yi\Y 1 - 1 J) (102a) 

n i 

= E~#(W) (102b) 

i=1 n 

n ] 

= (102c) 

i=l n 

< jr-T(D u ,D 2i ) (102d) 
i=1 ra 

^TjCX^Da) (102e) 

where (|102bt follows from the definition of W hi , (|102ct and (|102dt follow from ([38]) in Definition 
|3] and from Lemma [3] 

Achiev ability: Fix p (u>i, iu 2 |x, y). First generate Mi = 2 n( - I( - Wl]XY ^ +e \ W™ (ji) sequences, 
ji = 1,2, . . . , Mi, i.i.d. according to p(wi) (obtained from p (wi, w 2 \x, y)). Generate M 2 = 
2n{i(w 2 ;XYWi)+e) yyn sequences i.i.d. according to p (w 2 ) (obtained from p (wi, w 2 \x, y)). Bin 
the resulting W' 2 n sequences into S bins (analogously to the Wyner-Ziv binning), chosen at random 
where S = 2 n{ - I ( XYW ^ w ^- I ^ Y > w ^ +e \ and index these bins as b (j 2 ). Upon observing a source 
sequence (x n ,y n ) , the encoder searches for a W™ (ji) sequence such that (x n , y n , (ji)) E 
Txywx {n, e ) ( me choice of Mi ensures that there exists at least one such ji). Next, the encoder 
searches for a w 2 (j 2 ) such that (x n , y n , (ji) , w 2 (j 2 )) G TxyWiW 2 i n i e ) ( me choice of M 2 
ensures that there exists at least one such j 2 ). The encoder sends (ji,b(j 2 )) where b(j 2 ) is the 
bin index of the w 2 (j 2 ) sequence at a rate R = I(XY; W\) + /(X; M^lWiV) + e. Thus, we 
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have 

Pr (B = b (j 2 ) | (x n , y n , < (ji) , w n 2 (j 2 )) G Txy Wi w 2 (n, e)) 

= Pr (B = b (j 2 ) K (h) e 7V a (n, e)) = 1/5. (103) 

where (11031) is the result of the code construction which yields a Markov chain relationship 
(XYWi) -W 2 -B. With fji as defined in © for the typical set 7xyww 2 , and J = ( J 1 , B ( J 2 )), 
the achievable equivocation can be lower bounded as 

-H{Y n \J x ,B{J 2 )) 
n 

> -H(Y n \J u B(J 2 ),n) (104a) 
n 

= -H (Y n \W? (Ji) , 5 ( J 2 ) , /i) (104b) 

n 

> Pr (/i = 1) iff (Y n \W? (JO , 5 ( J 2 ) , /i = 1) . (104c) 

n 

The probability Pr (y n \w™ (ji) , b (j 2 ) , fJ, — 1) for all ji, j 2 , and y n can be written as 

EP^v n ,w^H(j 1 ),b(j 2 ),ii = i) 

= £ Pr KK (jO , 6 (j 2 ) , /i = 1) Pr (j,»K (ii) , A* = 1) (105a) 

where (I105al) follows from the fact that (XYW\)—W 2 —B forms a Markov chain. The probability 

Pr (w 2 \wi (ji) ,b(j 2 ) , fi = 1) can be rewritten as 

Pr «,< |/i = l) 



Pr « (ji), 6 (j 2 )|/i=l) 
Pr ( Jl )\ fi =l)/\S\ 



(106) 



EPrK,^(ii) |/x = l)/|5| 

UI 2 

= PrKK( Jl ),p = l). (107) 
Substituting (|107l) in (I105al) . Pr (y n \w™ , b (j 2 ) , // = 1) can be written as 

£ KK (Ji) , M = 1) Pr (V>? (Ji) , < A* = 1) 

= EPr(f,<K(ii),M-l) (108a) 

= Pr( 1 /»|<(j 1 ),^ = l) (108b) 
< 2 -»W y lW0-«) ( 108c) 
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where we have used the fact that for a typical set Px(7ywiW2 ( n 5 e )) — (1 — e ) [18, chap. 2]. 
From (1104cl) and (I108cl) . we then have 

* (>-K (*) , 6 fo) , 1) = £ Pr (yK (*) , /- 1) log Pr(m ^ l)>/X = 1) d09a) 

> £ Pr (y>? (Ji) , ^ = 1) n {H {Y\W X ) - e) (109b) 

>n(l-e)(H(Y\W 1 )-e) (109c) 

where in (|109b| ) we have used the fact that for a typical set Pr (T Y w 1 w 2 ( n > e )) > (1 ~ e ) [18, 
chap. 2]. Thus, we have 

-tf (Y n |J) > Pr 0* = 1) - £ Pr « (Ji) , b (j 2 ) \fi=l)H (Y n \w? ( 3l ) , 6 (j 2 ) , /i = 1) 

(110) 

> (1 - e) 3 (H (Y|Wi) - e) (111) 

where we have used the fact that for a typical set Pr (7yww 2 ( n > e )) > (1 — e ) [18, chap. 2]. 
The proof concludes by observing that H (Y n ) > H (Y n \J) and e — > as n — > oo. 

C. Converse Proof for region Q 4 

We start by a simple lower bound on the rate 

R > I(X, Y; W x ) + /(X; W 2 \W X , Y) 

>I{X,Y-X X ) (112) 

and an upper bound on T 

r < H(Y\W X ) 
= H(Y\W 1 ,X 1 ) 
< H{Y\X X ) 

= H{Y)-I{Y-X 1 ). (113) 

We will now use the distortion constraint of decoder 1 alone to simultaneously lower bound the 
rate and upper bound the equivocation. Consider an arbitrary p^'(xi\x,y) (and denote this as 
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distribution V\) given as: 

pW(0|0,0) = o, P {1) (0|1,1) =b 
p (1) (0|0,£) = c, p (1) (0|l,£) = d. 

For this distribution, we have 

P(X^Xx) = (i/2)[(l-p)(l-o + 6)+p(l-c + d)l (114) 
H(X X ) = h Q [(1 - p)(a + 6) + p(c + d)] ) (115) 

Y) = ^^-(h(a) + h(b)) + P -{h{c) + h{d)) (116) 
H{X X \Y) = i ^-{h{a) + h(b))+ph (S±^j . (117) 

These four quantities characterize the bounds in (II 121) and (|1 131) exactly and also the achievable 
distortion. 

Now consider a new distribution Vi, with conditional probabilities as follows: 

p (2) (0|0,0) = 1 — 6, p (2) (0|l,l) = 1 - a 
p (2) (0|0,£) = l-d, p (2) (0|l,£) = 1 - c. 

It is straightforward to verify that the distortion, rate and equivocation terms are the same for 
both V\ and P 2 . Next, define a new distribution P 3 as follows: 



p (3) (xi|x,?/) 



pW(fi|x,y) w.p. 1/2, 
p( 2 )(xi|x, y) w.p. 1/2. 

We now note that I(X,Y;Xi) is convex in p(xi\x,y) and #(Y|Xi) = H(Y) - I(Y;Xi) is 
concave in p{xi\y). By Jensen's inequality, this implies that the distribution P 3 defined above 
uses a rate that is at most as large and leads to an equivocation that is at least as large when 
compared to both the distributions V\ and Vi- Hence, it suffices to consider input distributions 
of the form p^(x x \x, y), which can be explicitly written as 

p( 3 )(0|0,0) = 1 -0, p (3) (0|l, 1) = /3 
p (3) (0|0,£) = 1 -a, p (3) (0|l,£) = a. 
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To satisfy the distortion constraint, we also have 

D 1 > (1 -p)(3 + pa 

which leads to /3 — (D 1 — pa)/(l — p). Now, also note that for a fixed a, this scheme yields a 
distortion of pa at the decoder 2. Furthermore, since the range of a £ [0, Di/p], we note that 
the worst case distortion for decoder 2 (for a fixed D{) is pD^/p = D x . This implies that as 
long as 

D 2 > D 1 

this region yields the stated tradeoff for the region Q 4 . 
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